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AUTOMATIC DIGITAL IMAGE ENHANCEMENT 



BACKGROUND OF THE INVENTION 
Field of the Invention 
5 The present invention relates to the field of digital image processing. 

More specifically, the present invention relates to a method and apparatus for 
enhancement of digital images. 

Related Art 

1 0 Digital cameras and scanners have become increasingly popular in 

recent years for generating digital images. Because of the recent decreases in 
cost for digital cameras and scanners, more and more photo pictures are 
generated, transmitted and saved in digital forms by amateur users instead of 
professionals. Unlike professional photos, photos produced by amateur users 

1 5 tend to suffer from poor image quality. These image quality problems result 
from under exposure, over exposure, poor lighting conditions, bad focus, 
inappropriate scanning processes, etc. 

For the above reasons, it is important to design ways to guarantee picture 
20 quality in a way that requires the least user input and that works mainly on itself, 
so that even amateurs can use it. The point-and-shoot camera is a good 
example of a product that is designed to meet this requirement. 

Digital processing methods typically focus on the picture capture stage, 
25 i.e., exposure setting, focus control, etc. The state of the art in digital camera 
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design has been quite successful by emulating traditional analog film cameras. 
However, camera design can not solve all of the quality problems because 
picture quality is partially a psychological problem. 

5 Digital post-processing methods have been designed that include image 

enhancement algorithms that improve image quality. Among recent image 
enhancement algorithms, a tone mapping algorithm is generally employed for 
adjusting the distribution of pixel values to produce better quality. The use of a 
tone mapping algorithm has been quite successful for satellite photos, airborne 

1 0 photos and medical images. For these applications, a major quality 

enhancement purpose is to illustrate the details of the images. However, these 
algorithms do not work well on consumer photos, because for consumer photos, 
the quality enhancement is more of a psychological problem than a 
mathematical problem, and detail illustration is not necessarily an important 

1 5 factor for quality evaluation. 

Recently, some computer programs have been developed for digital 
image enhancement for consumer pictures. An example of a successful 
program that provides digital image enhancement is Photoshop, a software 

20 product manufactured by Adobe. Though Photoshop includes a large 

repository of processing tools, it does not offer automatic processing. Users 
have to choose appropriate tools and parameters for their pictures based on 
their own judgment. Therefore, this is not a good tool for amateur users as they 
cannot easily determine the proper tools to use or the proper parameters to 

25 specify. 
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What is needed is a method and apparatus for enhancing a digital image 
that produces images that are perceived to be of good image quality. In 
particular, a method and apparatus for enhancing a digital image is needed that 
produces an enhanced digital image that is perceived to be of good image 
quality to amateur users. Also, what is needed is a method and apparatus that 
meets the above needs and that is automatic. Moreover, a method and 
apparatus is needed for enhancing a digital image that meets the above needs 
and that does not require user input. 
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SUMMARY OF THE INVENTION 

The present invention provides a method and apparatus for enhancing a 
digital image that is automatic and that does not require user input. In addition, 
the method and apparatus of the present invention produces an enhanced 
5 image that is perceived to be of good image quality. 



The majority of consumer photos include human faces. According to 
recent psychological studies, face region lightness plays an important role in 
picture quality evaluation. The method for enhancing a digital image of the 
1 0 present invention incorporates this psychological factor to produce an 
enhanced image that is perceived to be of good image quality. 



In one embodiment of the present invention, each digital image is 
converted from its original format into a format suitable for easy manipulation. 

15 In the present embodiment, the digital image is converted from its original 
format into CIE L *a*b* color space. Conversion of a digital image from its 
original format into L *a*b* color space as is more fully described in "Color 
Space Conversions," by Adrian Ford and Alan Roberts, September 1 996, which 
is available at www.inforamp.net/~poynton/PDFs/coloureq.pdf, which is 

20 incorporated herein, by reference, in its entirety. The resulting format includes a 
L* component that represents the lightness of each pixel of the image. The 
format also includes a* and b* components that represent the chromatic 
features of each pixel of the image. 
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The method for enhancing a digital image of the present invention 
locates human faces within a digital image. In the present embodiment, a face 
detection algorithm is used to detect human faces. The digital image is then 
analyzed, including analysis of the regions determined to be human faces and 
5 the analysis of the digital image as a whole. A tone mapping function is then 
determined for enhancing the image quality of the digital image. The tone 
mapping function is determined using both the analysis of detected faces and 
the analysis of the image as a whole to obtain a tone mapping function that 
incorporates both psychological factors (e.g., average face region lightness 
10 and average picture lightness) and signal factors (e.g., digital resolution, face 
region contrast, histogram uniformity, and noise issue). 

The tone mapping function is then applied to the digital image so as to 
produce an enhanced digital image. In one embodiment the tone mapping 

15 function is directly applied to each pixel element to obtain an enhanced digital 
image. In another embodiment of the present invention a lookup table is 
generated that represents the tone mapping function. The look up table is 
applied to the entire digital image to map an enhanced digital image. More 
particularly, in the present embodiment, each lightness (L*) component is 

20 altered according to the values stored in the lookup table to generate an 
enhanced digital image having altered tone values. The components that 
represent the chromatic features are not altered (i.e., a* and b* components are 
not changed). 
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The present invention is automatic. That is, the user needs only initiate 
operation of a program that performs the method of the present invention. All of 
the steps of the present invention are then performed automatically. Therefore, 
there is no need for a user to choose between a variety of tools to determine the 
5 appropriate tools to use as is required in prior art methods. 

The method and apparatus of the present invention does not require that 
the user provide any input. Therefore, there is no need for a user to determine 
various parameters that must be specified as is required by prior art methods. 

10 

Accordingly, the method and apparatus of the present invention is a good 
tool for amateurs to use as they will not be required to determine the proper 
tools to use or determine the proper parameters to specify as is required by 
prior art processes. In addition, the method and apparatus of the present 
1 5 invention is easy to use since the user needs take no action other than initiation 
of the program that is operable to perform the method of the present invention. 
Also, the method and apparatus of the present invention produces enhanced 
images that are perceived to be of good image quality. 

20 The present invention provides these advantages and others which will 

no doubt become obvious to those of ordinary skill in the art after having read 
the following detailed description of embodiments in accordance with the 
present invention. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a logical block diagram of an exemplary computer system in 
accordance with an embodiment of the present invention. 

5 Figure 2 is a diagram that illustrates a method for enhancing a digital 

image in accordance with one embodiment of the present invention. 

Figure 3 is a diagram that illustrates a method for enhancing a digital 
image that uses a look-up table and in which the original image is converted 
10 into L*a*b* space in accordance with one embodiment of the present invention. 

Figure 4 is a diagram that shows an exemplary two dimensional feature 
space in which the energy function for psychological evaluation is performed in 
accordance with an embodiment of the present invention. 

15 

Figure 5 is a diagram of a state machine depiction of the tone mapping 
function of the present invention based on nine-category classification of input 
images in accordance with an embodiment of the present invention. 

20 Figure 6 is a graph of an exemplary tone mapping curve in accordance 

with an embodiment of the present invention. 
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DETAILED DESCRIPTION OF THE INVENTION 
Reference will now be made in detail to the preferred embodiments of the 
invention, examples of which are illustrated in the accompanying drawings. 
While the invention will be described in conjunction with the preferred 
5 embodiments, it will be understood that they are not intended to limit the 

invention to these embodiments. On the contrary, the invention is intended to 
cover alternatives, modifications and equivalents, which may be included within 
the spirit and scope of the invention as defined by the appended claims. 
Furthermore, in the following detailed description of the present invention, 

1 0 numerous specific details are set forth in order to provide a thorough 

understanding of the present invention. However, it will be obvious to one of 
ordinary skill in the art that the present invention may be practiced without these 
specific details. In other instances, well known methods, procedures, 
components, and circuits have not been described in detail as not to 

1 5 unnecessarily obscure aspects of the present invention. 

Some portions of the detailed descriptions which follow are presented in 
terms of procedures, logic blocks, processing, and other symbolic 
representations of operations on data bits within a computer or digital system 

20 memory. These descriptions and representations are the means used by those 
skilled in the data processing arts to most effectively convey the substance of 
their work to others skilled in the art. A procedure, logic block, process, etc., is 
herein, and generally, conceived to be a self-consistent sequence of steps or 
instructions leading to a desired result. The steps are those requiring physical 

25 manipulations of physical quantities. Usually, though not necessarily, these 
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physical manipulations take the form of electrical or magnetic signals capable of 
being stored, transferred, combined, compared, and otherwise manipulated in a 
computer system or similar electronic computing device. For reasons of 
convenience, and with reference to common usage, these signals are referred 
5 to as bits, values, elements, symbols, characters, terms, numbers, or the like. 



It should be borne in mind, however, that all of these terms are to be 
interpreted as referencing physical manipulations and quantities and are 
merely convenient labels and are to be interpreted further in view of terms 

1 0 commonly used in the art. Unless specifically stated otherwise as apparent 
from the following discussions, it is understood that throughout discussions of 
the present invention, discussions utilizing terms such as "determining," 
"locating," "analyzing," "applying," "calculating," or the like, refer to the action 
and processes of a computer system, or similar electronic computing device, 

1 5 that manipulates and transforms data. The data is represented as physical 

(electronic) quantities within the computer system's registers and memories and 
is transformed into other data similarly represented as physical quantities within 
the computer system memories or registers or other such information storage, 
transmission, or display devices. 

20 

EXEMPLARY COMPUTER SYSTEM IN 
ACCORDANCE WITH THE PRESENT INVENTION 

Figure 1 is a block diagram of one embodiment of an exemplary 

25 computer system 100 used in accordance with the present invention. It should 

be appreciated that system 1 00 of the present embodiment is well suited to be 
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implemented as any of a number of different types of computer (e.g., portable 
laptop computer, desktop computer, server computer, personal digital assistant, 
palmtop computer and the like). Within the following discussions of the present 
invention, certain processes and steps are discussed that are realized, in one 
5 embodiment, as a series of instructions (e.g., software program) that reside 
within computer readable memory units of computer system 100 and executed 
by processor(s) of system 100. When executed, the instructions cause system 
1 00 to perform specific actions and exhibit specific behavior which is described 
in detail below. 

10 

In general, computer system 100 of Figure 1 , used by an embodiment of 
the present invention, comprises an address/data bus 110 for communicating 
information, one or more central processors 1 02 coupled with bus 1 1 0 for 
processing information and instructions. Central processor unit 102 may be a 

1 5 microprocessor or any other type of processor. The computer system 1 00 also 
includes data storage features such as a computer usable volatile memory unit 
104 (e.g., random access memory, static RAM, dynamic RAM, etc.) coupled with 
bus 1 10 for storing information and instructions for central processor(s) 102, a 
computer usable non-volatile memory unit 106 (e.g., read only memory, 

20 programmable ROM, flash memory, EPROM, EEPROM, etc.) coupled with bus 
1 10 for storing static information and instructions for processor(s) 102. System 
100 also includes an optional signal input and output device 108 coupled with 
bus 1 10 for enabling system 100 to interface with other electronic devices. In 
one embodiment of the present invention, the communication interface 108 is a 

25 serial communication port, but could also alternatively be any of a number of 
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well known communication standards and protocols, e.g., Universal Serial Bus 
(USB), Ethernet, FireWire (IEEE 1394), parallel, small computer system 
interface (SCSI), infrared (IR) communication, Bluetooth wireless 
communication, etc. 

5 

Optionally, computer system 100 can include a display device 112 which 
is coupled to bus 1 10 for displaying graphics and/or video. It should be 
appreciated that optional display device 112 may be a cathode ray tube (CRT), 
flat panel liquid crystal display (LCD), field emission display (FED), or any other 
10 display device suitable for displaying video and/or graphic images and 

alphanumeric characters recognizable to a user. Furthermore, system 100 can 
include an optional alphanumeric input device 114 including alphanumeric and 
function keys coupled to the bus 1 10 for communicating information and 
command selections to the central processor(s) 1 02. 

15 

Additionally, the computing device 100 of Figure 1 can include an 
optional cursor control or cursor directing device 116 coupled to the bus 1 10 for 
communicating user input information and command selections to the central 
processor(s) 102. The cursor directing device 116 can be implemented using a 

20 number of well known devices such as a mouse, a track-ball, a track pad, an 
optical tracking device, a touch screen, etc. Alternatively, it is appreciated that a 
cursor can be directed and/or activated via input from alphanumeric input 
device 114 using special keys and key sequence commands. The present 
invention is also well suited to directing a cursor by other means such as, for 

25 example, voice commands. In addition, the cursor directing device 116 can 
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aiso be a remote control device (e.g., a universal remote control device having 
a number of buttons, dials, etc.) with an infra-red signal communication 
capability. System 100 can also include a computer usable mass data storage 
device 1 1 8 such as a magnetic or optical disk and disk drive coupled with bus 
5 1 1 0 for storing information and instructions. 

METHOD FOR AUTOMATED DIGITAL IMAGE ENHANCEMENT 

1 0 Figures 2-3 show methods 200 and 300 that illustrate steps performed in 

accordance with embodiments of the present invention for automated digital 
image enhancement. Methods 200 and 300 include processes of the present 
invention which, in one embodiment, are carried out by processors and 
electrical components under the control of computer readable and computer 

1 5 executable instructions. The computer readable and computer executable 
instructions reside, for example, in data storage features such as computer 
usable volatile memory 104 and/or computer usable non-volatile memory 106 
of Figure 1 . However, the computer readable and computer executable 
instructions may reside in any type of computer-readable medium. In one 

20 embodiment of the present invention a computer readable medium is used to 
store instructions for performing the method of the present invention. In this 
embodiment the computer-readable medium is a magnetic or optical storage 
device such as a disk drive, magnetic tape, Compact Disk(CD), DVD (digital 
versatile disk or digital video disk). In yet another embodiment, the method of 

25 the present invention is performed using computer instruction signals embodied 
in a transmission medium (with or without a carrier wave upon which the 
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signals are modulated). For example, the transmission medium may include a 
communications network such as the Internet. 

Although specific steps are disclosed in methods 200 and 300 of Figures 
5 2-3, such steps are exemplary. That is, the present invention is well suited to 
performing various other steps or variations of the steps recited in Figures 2-3. 
Within the present embodiment, it should be appreciated that the steps of 
methods 200 and 300 may be performed by software or hardware or any 
combination of software and hardware. 

10 

As shown by step 201 of Figures 2-3, human faces are located within the 
digital image. In the present embodiment, a face detection algorithm is applied 
to the digital image for locating faces within the digital image. In one 
embodiment, a neural network based algorithm is applied to the digital image to 
1 5 identify any faces within the digital image. 

In the embodiment shown in Figure 2, the digital image can be 
manipulated using the format in which it is received or using any of a number of 
different common formats for analysis and manipulation of the digital image. 

20 

In the embodiment shown in Figure 3, each digital image is converted 
from its original format into L *a*b* color space as shown by step 310. For 
convenience, the present invention is described with reference to L*a*b* color 
space in the remainder of the detailed description. However, it is appreciated 
25 that other formats and corresponding tone mapping values could also be used. 
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As shown by step 202, the digital image is analyzed. In the present 
embodiment, the analysis includes analysis of the faces located in step 201 and 
the analysis of the digital image as a whole. 

5 

Continuing with step 202, in the present embodiment, the regions of the 
image that were identified to be human faces are analyzed to determine 
average face region lightness (L,). Average face region lightness can be 
determined by calculating the mean of the L* level of the human faces detected 
1 0 in the digital image. In the present embodiment, the human faces are also 
analyzed to determine face region contrast. 

In the present embodiment, the entire picture is analyzed to determine 
average picture lightness (L p ). The average picture lightness can be 
1 5 determined by calculating the mean of the L* level of the whole picture. The 
entire picture is also analyzed for determining digital resolution and histogram 
uniformity. 

A tone mapping function is then determined as shown by step 203 for 
20 enhancing the image quality of the digital image. In the present embodiment, 
the tone mapping function is a one-dimensional global function that maps one 
input value to only one output value. In one embodiment, the tone mapping 
function is based on the statistical measurement of the whole image as well as 
the measurement of face regions contained in the digital image. In the present 
25 embodiment, the face region(s) are treated as a region of interest and are 
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treated independently in the determination of the tone mapping function as 
compared to the image as a whole. 

In the present embodiment, the psychological factors of face region 
5 lightness and average picture lightness are used in addition to signal 

factors(digital resolution, face region contrast, histogram uniformity and noise 
issue) in determining the tone mapping function. 

The tone mapping function can be represented in graphical form by a 
10 tone mapping curve. In the present embodiment, the tone mapping curve is an 
integer-to-integer mapping curve in which both x and y integers are in the range 
of from 0 to 255. This conforms to the range of lightness (L) within a 
conventional L*a*b* format. However, other ranges can be used to 
accommodate different formats. 

15 

In the embodiment shown in Figure 3, the tone mapping function is 
determined (step 203) by first designing the tone mapping curve as shown by 
step 31 1 and using the tone mapping curve to generate a look-up table as 
shown by step 312. 

20 

As shown by step 204, the tone mapping function determined in step 203 
is applied to the digital image so as to produce an enhanced digital image. In 
the embodiment shown in Figure 3, the tone mapping function is applied to the 
digital image by mapping the L* values using the look-up table. More 
25 particularly , in the present embodiment, each L* value within the digital image 
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is replaced with a corresponding L* value from the lookup table. Thereby, the 
tone values of the original digital image are altered, producing an enhanced 
digital image. 



5 In the embodiment shown in Figure 3, the digital image is converted back 

into its original format. More particularly, the image is changed from the L*a*b* 
format back into the original format or "color space". 



In the present embodiment, steps 201 through 204 of Figures 2-3 and 
10 steps 310-314 of Figure 3 are performed automatically and without any user 
input. In one embodiment, the steps of methods 200 and 300 are performed by 
a computer program upon initiation of the computer program to perform the 
method of the present invention. The computer program then performs steps of 
methods 200 and 300 automatically, resulting in the creation of an enhanced 
15 digital image. In the present embodiment, the program does not require the 
user to provide any input or make any choices between different options to 
obtain the resulting output of an enhanced digital image. 



The tone mapping process of the present invention only alters the L* 
20 value, leaving the a* and b* components unchanged. This approach improves 
the lightness distribution of the digital image while maintaining the original 
chromatic information. 
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TONE MAPPING FUNCTION 
a. Psychological Factors 

Because human eyes adapt rapidly to the level of a relatively small angle 
in the visual field around the current fixation point, human eyes are only 
5 sensitive to relative luminance difference rather than absolute luminance 

difference. Accordingly, the psychological factors in the tone mapping function 
of the present are designed from two aspects, average face region lightness 
(L f ) and average picture lightness (L p ). 

1 0 It has been found that human observers prefer- to have both human face 

lightness L, and picture lightness L p to be mapped to specific optimal values 
optimal human face lightness (L° f ) and optimal picture lightness (L° p ). Human 
face lightness L, and picture lightness L p are coupled in influencing the 
subjective quality evaluation of an image. However, the two optimal values of 

15 optimal human face lightness L°, and optimal picture lightness L° p . can not 

always be obtained for a particular digital image. For example, one bright face 
may be captured with a dark background. 

It has been found that human evaluation of picture quality is pretty 
20 gradual with respect to the values of human face lightness L, and picture 

lightness L p . Therefore, some pictures still get good quality evaluation with their 
human face lightness L, and picture lightness L p close to the optimal values. 
The tone mapping function of the present invention takes advantage of this by 
using optimal ranges instead of optimal values. 

25 
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Figure 4 shows an example of an embodiment of the present invention in 
which optimal ranges are used for human face lightness L, and picture lightness 
L p . Along the axes of L, and picture lightness L p , their optimal region is defined 
as (L f low , L f high ) and (L p low , L p hish ) respectively. With optimal range definitions for 
5 human face lightness L f and picture lightness L p , pictures are classified into 
three categories: Dark, Normal and Bright along the two axes of human face 
lightness L, and picture lightness L p . When coupling these two parameters, 
pictures can be classified into nine categories in a two dimensional space. In 
the present embodiment, given a picture with parameters (L, L p ), psychological 
1 0 based quality evaluation is represented as an energy' term E ps (L f L p ). This gives 
a quality improvement problem that is formulated as an energy minimization (or 
maximization) problem in a two dimensional space as indicated by the following 
equation: 

mmE ps ( Lf ,L p ) (D 
15 To represent the coupled relationship between L f and picture lightness L p 

in determining the visual quality of the picture, the energy function E ps is defined 
as a two dimensional function of human face lightness L, and picture lightness 
L p individually over the nine sub-regions that correspond to the nine image 
categories of Figure 4. In the present embodiment, the energy function E ps is 
20 defined as follows: 

^M^ih-^T) 1 ^^-^) 1 - (2) 

where L/^ and L p 0(i) represent the optimal mean L* value for the face and picture 
regions if the input image has the (L, L p ) pair located within the i-th region of 
Figure 4. A human face lightness weighting factor (W, (i) ) and a picture lightness 
25 weighting factor (W p (i) ) are also used. Two different sets of optimal values (L,°, 
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L p °) are defined for human face lightness L, and picture lightness L p over the 
nine image categories/regions. For input images located within regions 1 to 8, 
both optimal human face lightness L, 0 , and optimal picture lightness L p ° are set 
to 120 (out of an 8-bit dynamic scale that runs from 0 to 255). However, images 
5 that fall within region 9, will have an optimal human face lightness L f °, and 
optimal picture lightness L p ° that are both set to 141 . In addition, the optimal 
region of both human face lightness L, and picture lightness L p are set as (100, 
140). 

10 Figure 5 illustrates an enhancement pattern produced by the present 

definition of quality evaluation when only psychological evaluation is 
considered. It can be seen that input pictures are divided into nine categories 
based on their picture level and ROI (face regions) statistics. With each 
category being represented as a state, the tone mapping function can be 

15 modeled as a state machine in that it tries to push the input picture from one 
state into another new state by mapping the lightness of the pixels. The 
introducing of a separate set of optimal parameters for pictures in region 9 will 
ultimately lead to two absorbing states: state 5 and state 9 for this state 
machine. These two absorbing states represent the convergence output of the 

20 present tone mapping algorithm if it is applied to an input image repeatedly. In 
practice, we find that this design helps better address human eye's sensitivity to 
the relative lightness and produces better visual results on these pictures with 
bright backgrounds. 

25 b. Signal Factors 
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Though psychological factors suggest that picture statistics human face 
lightness L, and picture lightness L p should be always mapped to their optimal 
values optimal human face lightness L f °, and optimal picture lightness L p °, in a 
global mapping scenario, Equation (2) can not be easily minimized because 
5 human face lightness L f and picture lightness L p are not independent. They are 
each based on picture signals and constrained by certain signal features. 
Therefore in the present embodiment, a number of signal factors are also 
included into consideration for tone mapping design. 

1 0 Digital resolution is a signal factor that is included in the present tone 

mapping function. For a 8-bit picture, it is desirable to make good use of 8-bit 
resolution. That is, the grayscale histogram of the picture should be distributed 
over the full range of values (in the present embodiment from 0 to 255). In the 
present embodiment, the concept of shadow L s and highlight L h are used to 

15 compensate for digital resolution. Shadow L s and highlight L h both represent 
the grayscale boundary that clips the darkest (or brightest) 0.5 percent of the 
picture pixels. If shadow L s is greater than zero or highlight L h is less than 255, 
a linear stretch is applied to map shadow and highlight to the scale of from 0 to 
255 to get full digital resolution of the picture. 

20 

Face region contrast, represented by the greyscale standard deviation of 
the face regions (8), is a signal factor that is included in the present tone 

mapping function. Since human faces are considered as a region of interest, it 
is also desirable to use more digital grayscale resolution for face regions. 
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Therefore, in the present embodiment, face region contrast 8 is always 
increased when possible. 

Histogram uniformity is a signal factor that is also included in the present 
5 tone mapping function. Though extreme uniformity of histogram (i.e., histogram 
equalization) is not desirable for consumer photos, uniformity of the histogram 
distribution is still an important factor from signal presentation point of view. 
Therefore, it is considered as a signal constraint. 

1 0 Noise issue is another signal factor that is included in the present tone 

mapping function. The tone mapping function is an integer to integer mapping 
function that can be represented by a mapping curve. A large slope of the 
mapping curve tends to magnify small image noise, and is therefore a negative 
factor that is included in the present tone mapping function. 

15 

c. Tone Mapping Curve Design 

The tone mapping function of the present invention is designed to cover 
both psychological and signal factors. Because psychological factors and 
signal factors are different semantics, in the present embodiment, their 
20 independent influences are combined with a system energy formulation. That 
is, the desirable tendency of the psychological and signal constraints are each 
formulated as their respective energy terms, psychological energy (E ps ) and 
signal factor energy (E signal ). The overall optimization of the tone curve design is 
then expressed as an energy minimization problem of the system energy term 
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Min(E) = min(E ps + E signal ). Here the minimization of the system energy is 
actually to find appropriate trade-offs between multiple constraints that 
contribute from different aspects to the same problem. 

5 d. Parametric Curve Representation 

The tone mapping function T: x-> y is an integer-to-integer mapping curve 
with both its input and output defined in the range of 0 to 255. To 
accommodate the mapping control of human face regions, the mapping function 
is designed as a two-segment gamma curve, an example of which is depicted in 

1 0 Fig 6. In Fig 6, axis x represents the input and axis y -represents the output 

grayscale respectively. L s and L h are the shadow and the highlight of the input 
image histogram and L af and L mf are the actual mean L* and the mapped mean 
L* of the face region. For simplicity of discussion, the control pair (L af and L mf ) is 
denoted as a control point (x 0 , y 0 ) (i.e., L a{ = x D and L m( = y 0 ). The parametric 

1 5 curve of the present invention then becomes: 




(3) 



20 




To decide the two gamma values a t and a 2 , we require that the two 



segments be first order smooth at the control point (x 0 , y 0 ). That is, their first 
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order derivatives at this point are equal, if we denote this derivative as y', then 
we have the equations: 

a x =^^-y (5) 

y 0 

L h -x n 

* 2 =— (6) 
Therefore, the tone mapping curve as defined in Equations. (3)-(6) is fully 
decided by three parameters (x 0 , y 0 , y). The design work is then to decide these 
three parameters based on the psychological and signal factors. 



e. Determination of y' 
1 0 In the present embodiment, y is the slope of the curve at (x OJ y 0 ). It 

therefore controls the grayscale contrast of the mapped face region. If we 
denote the standard deviation of the L* signal of the face region before the tone 
mapping as 5 a , and denote the standard deviation of the L* signal of the face 

region after the tone mapping as S mf , then the slope is y = jr- . 

15. 

As detected face regions are considered as the region of interest, it is 
always desirable to increase the mapped deviation of the face regions 8 mf as 

long as it does not interfere with the grayscale distribution balance of the 
mapped picture. Therefore, we require that the mapped deviation of the face 
20 region be greater than the deviation of the face region of the original digital 
image ( 8 mf > 5 af ). 

In the present embodiment, the mapped deviation of face region is 
determined as follows. First, the histogram density at the neighborhood of L a{ is 

HP-10011098 



-24- 

measured over the histogram of the original picture using the following 
equation: 

y(VW3) hi t(i) 
Density. = ^Yf {) (?) 

A histogram density ratio is then obtained by comparing this histogram density 
with the average histogram density of the picture using to the following 
equation: 

Density af 
k } — 



' width- height 1 255 (8) 
The mapped deviation of the face region is then decided based on histogram 
1 0 density ratio and a constant magnifying factor of 1 .2 according to the equation: 
8 mf = \.2-k-8 af (9) 

Accordingly, in the present embodiment, a balance between histogram 
uniformity and better face region contrast is obtained. Considering the 
1 5 consistency in the basic features of face regions, to avoid overdoing the contrast 
enhancement, we also set up an upper bound of S mf . This is, if 5 mf produced 

from Equation (9) makes 8 mf > A max , we set 8 mf = A max . After 8 mf is determined, the 
slope y' is simply set as y = ^ . 

20 f. Determination of (x 0 , y 0 ) 

The control point (x 0 , y 0 ) is represented by the control pair (L a{ , L mf ). In the 
present embodiment, L af is obtained directly from face detection. If more than 
one face is detected, L a( is set as the weighted mean of the means of all the 
faces (weighted by the area of the faces). 
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In the present embodiment, the mapped mean L* of face regions L mf is 
determined based on system energy minimization as: 

L mf = y 0 = Mgmin(E ps (y) + E signal (y)) ( 1 °) 

In the present embodiment, E ps is defined using Equation (2). Referring now 
back to Equation (2), L f = y and L p is a function of y, once the slope y' is 
determined. Therefore, E ps is a function of y while E signal in Equation (10) is 
further decomposed into the following energy terms: 

In Equation (11), E uniformity represents the histogram uniformity of the 
mapped image: 



»G0 = 



1, if >i (12) 



/^ 0 )(255-y) ' ^(x 0 )(255-y) 
M*,X255-y) otherw . se 



where f h( x o) = ^l 0 hist i x ) and h( x o) = Y^l LaI hist ( x ) and where hist(x) 
is the histogram of the input picture before tone mapping. 



In the present embodiment, the model energy term E model in Equation (11) 
20 is defined as: 

E mO M(y) = D-(y-x 0 f (13) 
where D is a weighting factor. This term is mainly intended to address the noise 
issue. That is, if the change in grayscale mapping is big, the slope will also be 
big and the noises on the original image tend to get magnified on the mapped 
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image. The weighting factor D compensates for magnificaton of the noises on 
the original image when the change in grayscale mapping is large. 

The foregoing descriptions of specific embodiments of the present 
5 invention have been presented for purposes of illustration and description. 
They are not intended to be exhaustive or to limit the invention to the precise 
forms disclosed, and obviously many modifications and variations are possible 
in light of the above teaching. The embodiments were chosen and described in 
order to best explain the principles of the invention and its practical application, 
1 0 to thereby enable others skilled in the art to best utilize the invention and 

various embodiments with various modifications as are suited to the particular 
use contemplated. It is intended that the scope of the invention be defined by 
the Claims appended hereto and their equivalents. 
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